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Abstract 

Discrete optimisation problems arise in many different areas and are studied under many different 
names. In many such problems the quantity to be optimised can be expressed as a sum of functions 
of a restricted form. Here we present a unifying theory of complexity for problems of this kind. We 
show that the complexity of a finite-domain discrete optimisation problem is determined by certain 
algebraic properties of the objective function, which we call weighted polymorphisms. We define a 
Galois connection between sets of rational-valued functions and sets of weighted polymorphisms and 
show how the closed sets of this Galois connection can be characterised. 

These results provide a new approach to studying the complexity of discrete optimisation. We use 
this approach to identify certain maximal tractable subproblems of the general problem, and hence 
derive a complete classification of complexity for the Boolean case. 

Keywords: Galois connection, valued constraint satisfaction problems, constraint optimisation, weighted 
polymorphisms, weighted clones, complexity 



1 Introduction 



Discrete optimisation problems arise in many different areas and are studied under many different names, 
including Min-Sum Problems, Gibbs energy minimisation, Markov Random Fields, Conditional Random 
Fields, 0/1 integer programming, pseudo-Boolean function minimisation, constraint optimisation and val- 
ued constraint satisfaction [Ji, 10, 40, 39, 17, 16, 7, 6]. 

*Part of this work (by D. A. Cohen, M. C. Cooper, and P. G. Jeavons) appeared in Proceedings of the 12th International 
Conference on Principles and Practice of Constraint Programming (CP), pp. 107-121, 2006. Part of this work (by D. A. Cohen, 
P. Creed, P. G. Jeavons, and S. Zivny) appeared in Proceedings of the 36th International Symposium on Mathematical Foundations 
of Computer Science (MFCS), pp. 231-242, 2011. Part of this work (by P. Creed and S. Zivny) appeared in Proceedings of the 
17th International Conference on Principles and Practice of Constraint Programming (CP), pp. 210-224, 201 1. Part of this work 
appeared in Stanislav Zivny's doctoral thesis at the University of Oxford, 2009. This research was supported by EPSRC grant 
EP/FOl 161X/1. Stanislav Zivny is supported by a Junior Research Fellowship at University College, Oxford. 
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Here we adopt a very general framework where each problem instance is specified by a set of variables, 
a set of possible values for those variables, and a set of constraints. Each combination of values allowed by 
each constraint has an associated cost, and the goal is to find an assignment with minimal total cost. This 
simple abstract mathematical framework can be used to express discrete optimisation problems arising 
in a wide variety of fields, including operational research (scheduling, resource utilisation, transportation), 
computer vision (region segmentation, object recognition, image enhancement), automated reasoning (Max 
SAT, Min ONES), graph theory (Min-Cut, Maximum Independent Set), and many others. 

In the special case when all costs are zero, the problem we are studying collapses to the standard con- 
straint satisfaction problem (CSP). The general CSP is NP-hard, and so is unlikely to have a polynomial- 
time algorithm. However, there has been much success in finding tractable fragments of the CSP by restrict- 
ing the types of relation allowed in the constraints. A set of allowed relations has been called a constraint 
language [ ]. For some constraint languages the associated constraint satisfaction problems with con- 
straints chosen from that language are solvable in polynomial-time, whilst for other constraint languages 
this class of problems is NP-hard [28, 27, 22]; these two cases are referred to as tractable languages and 
NP-hard languages, respectively. Dichotomy theorems, which classify each possible constraint language 
as either tractable or NP-hard, have been established for languages over 2-element domains [34], 3-element 
domains [13], for conservative languages [15, 1], and maximal languages [1, 12]. 

The more general framework we consider here, which allows non-zero costs, is also NP-hard, but again 
we can try to identify tractable fragments by restricting the types of allowed constraints. Each type of 
constraint is specified by a rational- valued function defined on a set of tuples, which specifies the cost asso- 
ciated with each allowed tuple of values. Such a function is called a weighted relation, and a set of weighted 
relations will be called a valued constraint language [ ]. Much less is known about the complexity of the 
optimisation problems associated with different valued constraint languages, although some results have 
been obtained for certain special cases. In particular, a complete characterisation of complexity has been 
obtained for valued constraint languages over a 2-element domain [K ]. This result generalises a number 
of earlier results for particular optimisation problems such as Max-Sat [18] and Min-Ones [ ]. A 
complete classification has also been obtained for valued constraint languages containing all unary {0, 1}- 
valued weighted relations (such languages are called conservative) [ ]. This result generalises a number 
of earher results such as Graph Min-Cost-Hom [25] and Digraph Min-Cost-Hom [ ]. 

One class of weighted relations that has been extensively studied and shown to be tractable is the class 
of submodular functions [ , , , ' , 30, 41]. 

In the classical CSP framework it has been shown that the complexity of any constraint language over 
any finite domain is determined by certain algebraic properties known as polymorphisms [28, 27]. This 
result has reduced the problem of the identification of tractable constraint languages to that of the identifi- 
cation of suitable sets of polymorphisms. In other words, it has been shown to be enough to study just those 
constraint languages which are characterised by having a given set of polymorphisms. Using this algebraic 
approach, considerable progress has now been made towards a complete characterisation of the complexity 
of constraint languages over finite domains of arbitrary size [22, 14, 4, 2, 3, . ]. 

In this paper, we introduce a new algebraic construct which we call a weighted polymorphism. We are 
able to show that the weighted polymorphisms of a valued constraint language are sufficient to determine 
the complexity of that language. In addition, we are able to define a Galois connection between valued 
constraint languages and sets of weighted polymorphisms, and characterise the closed sets on both sides. 

The Galois connection we establish here can be used in the search for tractable valued constraint lan- 
guages in a very similar way to the use of polymorphisms in the search for tractable constraint languages 
in the classical CSP. First, we need only consider valued constraint languages characterised by weighted 
polymorphisms. This greatly simplifies the search for a characterisation of all tractable valued constraint 
languages. Second, we will show below that any tractable valued constraint language must have an associ- 
ated non-trivial weighted polymorphism. Hence the results of this paper provide a powerful new set of tools 
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for analysing the complexity of finite-domain discrete optimisation problems. In fact, a recent result on the 
power of linear programming for valued constraint languages builds on weighted polymorphisms 

The structure of the paper is as follows. In Section 2 we introduce the general framework of the Valued 
Constraint Satisfaction Problem (VCSP) and define the notion of expressibility. In Section 3 we focus on the 
classical Constraint Satisfaction Problem and show how it fits in the VCSP framework as a special case. We 
briefly recall the notion of polymorphism, and the Galois connection that has been so fruitful in the study of 
the complexity of the classical Constraint Satisfaction Problem. In Sections 4 and 5 we introduce weighted 
relational clones (valued constraint languages closed under expressibility and certain other operations) and 
the corresponding closed sets of weighted polymorphisms, which we call weighted clones. We then state 
our main result: weighted relational clones are in 1-to-l correspondence with weighted clones. In Section 6 
we give proofs of the theorems establishing this Galois connection. In Section 7 we use the algebraic theory 
to establish necessary conditions that must be satisfied by any tractable valued constraint language. Using 
these results, we obtain a complete classification for the Boolean case in Section 8. Finally, in Section 9, 
we state some conclusions and outline directions for future work. 



2 The Valued Constraint Satisfaction Problem 

For any set D, the set of tuples of length r over D is denoted by D^, and a subset of is called a relation 
over D of arity r. A (partial) function g from to Q associates a rational' weight with each of the tuples 
in some subset of D^', and so will be called a weighted relation on D of arity r. 

The idea of a weighted relation is very general, and can be used to define a wide variety of discrete 
optimisation problems. The general framework we shall use for such problems is defined as follows. 

Definition 2.1. An instance of the valued constraint satisfaction problem, (VCSP), is a triple V = 
{V,D, C) where: V is a finite set o/ variables; D is a finite set of possible values; C is a finite multi- 
set o/ constraints. Each element ofC is a pair c = {a, g) where a is a tuple of variables called the scope 
ofc, and g is a weighted relation on D of arity \a\. 

An assignment /or V is a mapping s : V ^ D. The cost of an assignment s, denoted Cost-p{s), is 
given by the sum of the weights assigned to the restrictions of s onto each constraint scope, that is, 

Costj){s) ^= ^ g{s{vi),s{v2),. . . ,s{Vjn))- 

{{vi,v2,...,vm),g)eC 

If g{s{vi), s{v2), • • • , s{vm)) is undefined for some {{vi,V2, • • • , Vm) , g) S C, then the assignment s is 
said to be infeasible and Cost'p{s) is undefined. 

A solution to V is a feasible assignment with minimal cost. 

Example 2.2 (Min-Cut). In graph theory a cut of a graph is a partition of the vertices into two disjoint 
sets. The size of a cut is the number of edges of the graph that intersect both sides of this partition. The 
MiN-CuT problem for a graph is to find a cut with the smallest possible size. This problem can be solved 
in polynomial-time [32 ]. 

The MiN-CUT problem for the graph {V, E) can be expressed as an instance {V, {0, 1}, C) of VCSP 
by setting C = {{e, g=) | e G E}, where g= is the binary weighted relation on the set {0, 1} defined by: 

Q=Kx,y) < ^ otherwise ' 



' To avoid representational issues, we restrict ourselves to rational rather than real-valued weights. The resulting framework is 
sufficiently general to encode very many standard optimisation problems; for examples, see [16]. 
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Any assignment for this VCSP instance partitions the elements of V into those assigned the value and 
those assigned the value 1. The cost of the assignment is equal to the size of the corresponding cut. 

Example 2.3 (Max-Cut). The Max-Cut problem for a graph is to find a cut with the largest possible 
size. This problem is NP-hard [^j]. 

The Max-Cut problem for the graph [V, E) can be expressed as an instance iy, {0, 1}, C) of VCSP 
by setting C = {(e, q-^) \ e € E}, where is the binary weighted relation on the set {0, 1} defined by: 



def f if X / y 
1 otherwise 



Example 2.4 (DIGRAPH MIN-COST-HOM). Given two directed graphs (digraphs) G = {VgiEg) and 
H = (Vff, Efj), a homomorphism from G to H is a mapping f : Vq — > Vh that preserves edges, that is, 
(u, v) G Eq implies (/(u), f{v)) € Eh- Assume that for any u €z Vg and v € Vh a rational cost Cu{v) is 
given. The cost of a homomorphism f from G to H is then defined to be Y^usiVc ^"^ 

(/(u)). r/ie Digraph 

MiN-COST-HOM problem is to find a homomorphism from G to H of minimum cost [25, 37]. 

Given a fixed digraph H = (Vh, Eh), we denote by qh the binary weighted relation on the set Vh 
defined by: 

def / if (x, y) e Eh 
oo otherwise 



The Digraph Min-COST-HOM problem for input graph G = {Vq, Eg) and fixed target graph H = 
(VhtEh) can be expressed as an instance (Vg, Vh, C) of VCSP by setting C = {{e, qh) \ e G Eg} U 
{{u,Cu) I u G Vg}. 

A valued constraint language is any set T of weighted relations on some fixed set D. We define 
VCSP(r) to be the set of all VCSP instances in which all weighted relations in all constraints belong to T. 

Example 2.2 shows that VCSP({£)=}) includes the MiN-CUT problem, and Example 2.3 shows that 
VCSP({£»^}) includes the Max-Cut problem. In fact it is easy to show that VCSP({^=}) corresponds 
to the MiN-CuT problem in the sense that not only does VCSP({£i=}) include the MiN-CuT problem, but 
also any instance from VCSP({iJ=}) can be reduced to an instance of the MiN-CuT problem. Similarly, 
VCSP({^^}) corresponds to the Max-Cut problem. The problem VCSP({^=, g^}) includes both MlN- 
CUT and Max-Cut, as well as many other problems which can be expressed using these two types of 
constraints. 

A valued constraint language T is called tractable if, for every finite subset T' C F, there exists an 
algorithm solving any instance V € VCSP(r') in polynomial time. Conversely, T is called NP-hard if 
there is some finite subset P' C P for which VCSP(P') is NP-hard. Example 2.2 shows that the valued 
constraint language is tractable, and Example 2.3 shows that the valued constraint language {g^} is 
NP-hard. 

We now define a closure operator on weighted relations, which adds to a given set of weighted relations 
all other weighted relations which can be expressed using that set, in the sense defined below. 

Definition 2.5. For any VCSP instance V = {V, D, C), and any list L = {vi, . . . , Vr) of variables ofV, 
the projection ofV onto L, denoted iriiV), is the weighted relation on D ofarity r defined as follows: 

TTL{V){xi,...,Xr) =^ min Cost-p(s). 

{s:V~¥D I (s{vi),...,s{Vr)) = {xi,...,Xr)} 

We say that a weighted relation g is expressible over a valued constraint language P if there exists a VCSP 
instance V G VCSP(P) and a list L of variables ofV such that ULiV) = g. We call the pair {V,L) a 
gadget /or expressing g over P. 

We define Express(P) to be the expressive power o/P; that is, the set of all weighted relations ex- 
pressible over P. 
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Note that the Ust of variables L in a gadget may contain repeated entries, the sum over an empty set is 
zero, and the minimum over an empty set is undefined. 

Example 2.6. Let V be the VCSP instance with a single variable v and no constraints, and let L = {v, v). 
Then, by Definition 2.5, 



I undefined otherwise 

Hence for any valued constraint language T, over any set D, Express(r) contains this binary weighted 
relation, which will be called the weighted equality relation. 

Example 2.7. Let V be the VCSP instance with domain {0, 1}, variables vi,V2,v^, and constraints {{vi,V2) ,Q^) 
and {{v2, V3) , Q^), and let L = {vijV^). Then, by Definition 2.5, 



Hence iTLi'P) = Q=, so we have that q= E Expressd^j^}). 

However, using the results of this paper, we will be able to show, for example, that Express ({£>=}) 
(see Example 8.4). 

The next result shows that expressibility preserves tractability. 

Theorem 2.8. A valued constraint language T is tractable if and only //'Express(r) is tractable; similarly, 
r is NP-hard if and only //'Express(r) is NP-hard. 

Proof. By the definition of a tractable valued constraint language, it is sufficient to show that for any finite 
subset r' of Express(r) there exists a polynomial-time reduction from VCSP(r') to VCSP(r"), where 
r" is a finite subset of F. 

Let r' be a finite subset of Express(r) and let V' be any instance of VCSP(r'). By Definition 2.5, 
any weighted relation q' G Express(r) can be constructed by using some gadget {Vqi,L^ where "P^/ is an 
instance of VCSP(r). Hence we can simply replace each constraint in V' which has a weighted relation 
q' not already in F with the corresponding gadget to obtain an instance V of VCSP(F) which is equivalent 
to V' . The maximum size of any of the gadgets used is a constant determined by the finite set F', so this 
construction can be carried out in polynomial time in the size of V' . □ 

This result shows that, when trying to identify tractable valued constraint languages, it is sufficient 
to consider only languages of the form Express(F). In the following sections, we will show that such 
languages can be characterised using certain algebraic properties. 

3 Classical Constraint Satisfaction 

In this section we consider the special case when the weights are all zero. 

Definition 3.1. We denote byYi^o the set of all zero-valued weighted relations on a set D. 

There is a one-to-one correspondence between the set of zero- valued weighted relations R/j and the set 
of all relations over D. In this correspondence each weighted relation £i in R/j is associated with the relation 
R{q) containing precisely those tuples on which g is defined. Similarly, each zero-valued weighted relation 
in R/) is associated with the predicate which is true for precisely those tuples where the weighted relation 
is defined. Subsets of R/j are sometimes referred to as crisp constraint languages [16] and VCSP(Rd) is 
equivalent to the classical constraint satisfaction problem, or CSP, where each assignment is either allowed 
(cost 0) or disallowed (infeasible, or cost undefined). 




if X = y 
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Definition 3.2. A weighted relation g of arity r can be obtained by addition /rom the weighted relation qi 
ofarity s and the weighted relation Q2 of arity t if g satisfies the identity g{xi, . . . , Xr) = gi{yi, • • • , Us) + 
g2{zi, . . . , zt),for some (fixed) choice ofyi, ■ ■ ■ ,ys and zi, . . . , ztfrom amongst the xi, . . . , Xr- 

For zero-valued weighted relations this notion of addition corresponds to performing a relational join 
operation on the associated relations R{gi) and R{g2) [ ]■ It also corresponds to taking a conjunction of 
the associated predicates [14]. Moreover, minimising a weighted relation ^) € Rd over one of its arguments 
corresponds to taking a relational projection of R{g) onto its remaining co-ordinates. It also corresponds 
to existential quantification of the associated predicate over that argument. 

Definition 3.3. A setV C Yio is called a relational clone if it contains the weighted equality relation and 
is closed under addition and minimisation over arbitrary arguments. 

For each T C Yio we define RelClone(r) to be the smallest relational clone containing T. 

It is a straightforward consequence of Definitions 2.5 and 3.3 that the expressive power of a crisp 
constraint language is given by the smallest relational clone containing it, as the next result indicates. 

Proposition 3.4. For any T C R^), Express(r) = RelClone(r). 

This alternative characterisation for the expressive power of a crisp constraint language was first ob- 
served in [ ], and used to study the complexity of such languages using tools from universal algebra. We 
now give a brief summary of this algebraic approach. 

For any set D, a function / : D is called a A:-ary operation on D. 

(k) 

Definition 3.5. We denote by Od the set ofallfinitary operations on D and by O^, the k-ary operations 
in Od- 

Definition 3.6. The k-ary projections on D are the operations ef'^ : D , (oi, . . . , a^) 1— )■ aj . 

(k) (£) 

Definition 3.7. Let f € Oj^ and gi,. . . ,gk S O}, . The superposition of f and gi,. . . ,gk is the i-ary 
operation f[gi,.. .,gk]:D^^D, (xi,. . .,Xi)^ f{gi{xi, ■ ■ ■ . . .,gk{xi . . .,xe)) ■ 

Definition 3.8. A set F Q Od is called a clone of operations if it contains all the projections on D and is 
closed under superposition. For each F C Ojj we define Clone(F) to be the smallest clone containing F. 

We can extend A;-ary operations to operate on tuples in a natural way, as follows. Let xi, . . . ,Xfc be 
tuples of length r over a set D, where each Xj = (xj 1, Xj^2, ■ ■ ■ , ^i.r)- We can obtain another element of 
D"^ by applying / to the tuples Xi co-ordinatewise, as follows: 

/(Xi, . . . , Xfc) =^ {f{xi,U Xk^i),f{xi^2, • • • , Xk,2), • • • , f{xi,r, • • • , Xk,r)) ■ 

(k) 

Definition 3.9. Let g be a weighted relation of arity r on a set D and let f € O^, . We say that f is a 
polymorphism of g if for any xi, X2, . . . , x^ G such that g{xi) is defined for i = 1, . . . , k, we have 
that £»(/(xi, X2, . . . ,Xk)) is also defined. 

If f is a polymorphism of g we say g is invariant under f. 

Definition 3.10. For any valued constraint language T over a set D, we denote by Pol(r) the set of all 

operations on D which are polymorphisms of all weighted relations g G T and by Pol*^'^^ (F) the k-ary 
operations in Pol(r). 

Definition 3.11. For any F C Oj^, we denote by lnv{F) the set of all weighted relations in R/j that are 
invariant under all operations f € F. 
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For any set D, the mappings Pol and Inv form a Galois connection between Od and Ru [9]. A 
characterisation of this Galois connection for finite sets D is given by the following two theorems, originally 
obtained for sets of relations ' , 8]. 

Theorem 3.12. For any finite set D, and any finite T C R^j, Inv(PoI(r)) = RelClone(r). 

Theorem 3.13. For any finite set D, and any finite F Q Od, Pol(Inv(F)) = Clone(F). 

As with any Galois connection [ ], this means that there is a one-to-one correspondence between clones 
and relational clones. Together with Proposition 3.4, this result shows that the expressive power of any crisp 
constraint language P on a finite set D corresponds to a particular clone of operations on D. Hence, by 
Theorem 2.8, the search for tractable crisp constraint languages corresponds to a search for suitable clones 
of operations [27, 14]. This key observation paved the way for applying deep results from universal algebra 
in the search for tractable constraint languages [11, 15, 12, 13, 4, 2, 3, 5]. 

4 Weighted Relational Clones 

In this section we return to the general case of weighted relations taking arbitrary values in Q in order to 
define the notion of a weighted relational clone. 

(r) 

Definition 4.1. We denote by ^d the set of all weighted relations on D taking values in Q and by the 
weighted relations in ofarity r. 

We now define a closure operator on weighted relations, which adds to a set of weighted relations all 
other weighted relations which can be obtained from the original set by non-negative scaling and addition 
of a constant. 

Definition 4.2. A weighted relation g' ^ can be obtained from a weighted relation g € by 
non-negative scaling and addition of constants if there exist a, /3 G Q with a > such that g' = ag + P . 
We denote by P^ the smallest set of weighted relations containing P which is closed under non-negative 
scaling and addition of constants. 

The next result shows that adding weighted relations that can be obtained by non-negative scaling and 
addition of constants preserves tractability. 

Theorem 4.3. A valued constraint language P is tractable if and only if P^ is tractable; similarly, P is 
NP-hard if and only ifT^ is NP-hard. 

Proof. By the definition of tractable valued constraint language, it is sufficient to show that for any finite 
subset P' of P^ there exists a polynomial-time reduction from VCSP(P') to VCSP(P"), where P" is a 
finite subset of P. 

Let P' be a finite subset of P^ and let V' be any instance of VCSP(P'). By Definition 4.2, any weighted 
relation g' G P^ can be obtained by non-negative scaling and addition of constants from some weighted 
relation g £ T. Hence we can replace each of the constraints (cr, g') in V' with a new constraint (cr, g), 
where g and g' = ag + /3 for some non-negative rational value a and some arbitrary rational constant 
f3, to obtain an instance V of VCSP(P). 

The constant /3 is added to the cost of all assignments and so does not affect the choice of solution. 

Since a is a non-negative rational value, it can be expressed asp/q for some non-negative integer p and 
positive integer q. 

If p is non-zero, then the effect of the scale factor p/q can be simulated by taking p copies of the 
new constraint in V and q copies of all other constraints in V. The maximum values of p, q are constants 
determined by the finite set P', so this construction can be carried out in polynomial time in the size of V'. 
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It only remains to deal with the cases where p is zero. Assume that V' contains k constraints of the form 
{a, q') where q' = + /3, and replace each of these with the corresponding constraint (cr, g) to obtain a 
new instance, V. Let M be the maximum weight assigned by any weighted relation in the finite set T' , and 
let m be the minimum difference between any two distinct weights assigned by weighted relations in T' . 
The cost of any feasible assignment for V is greater by at most kM than the cost of the same assignment 
for V' . Hence if we also take + 1] copies of all the remaining constraints of V' , then we obtain an 
instance of VCSP(r) with the same solutions as V' . Since M and m are constants determined by the finite 
set r', this construction can again be carried out in polynomial time in the size of V' . □ 

Definition 4.4. A set T C is a weigiited relational clone if it contains the weighted equality rela- 
tion and is closed under non-negative scaling and addition of constants, addition, and minimisation over 
arbitrary arguments. 

For each F C we define wRelClone(r) to be the smallest weighted relational clone containing V. 

It is a straightforward consequence of Definitions 2.5, 4.2 and 4.4 that, for any valued constraint lan- 
guage r C the set of weighted relations that can be expressed using weighted relations obtained from 
r by non-negative scaling and addition of constants, is given by the smallest weighted relational clone 
containing F, as the next result indicates. 

Proposition 4.5. For any T C Express(r^) = wRelClone(r). 

Hence, by Theorem 2.8 and Theorem 4.3, the search for tractable valued constraint languages corre- 
sponds to a search for suitable weighted relational clones. 

In the next section we establish an alternative characterisation for weighted relational clones which 
facilitates this search. 

5 Weighted Clones 

To obtain a suitable alternative characterisation for weighted relational clones we now generalise the notion 
of a clone of operations, introduced in Definition 3.8, by introducing the notion of a weighted clone. 

Recall that a clone of operations, C, is a set of operations on some fixed set D that contains all projec- 
tions and is closed under superposition. The A:-ary operations in a clone C will be denoted C'^^\ 

Definition 5.1. We define a k-ary weighting of a clone C to be a function uj : C^''^ — )• Q such thatuj{f) < 
only if f is a projection and 

E ^(/) = o. 

(k) 

We denote by Wc the set of all possible weightings of C and by the set of k-ary weightings of C. 

For any weighting uj, we denote by dom(a;) the set of operations on which uj is defined. We denote by 
ar(a;) the arity of uj. 

Since a weighting is simply a rational- valued function satisfying certain inequalities it can be scaled by 
any non-negative rational to obtain a new weighting. Similarly, any two weightings of the same clone of 
the same arity can be added to obtain a new weighting of that clone. 

The notion of superposition from Definition 3.7 can also be extended to weightings in a natural way, by 
forming a superposition with each argument of the weighting, as follows. 
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Definition 5.2. For any clone C, any uj G W^^^-* and any gi,g2, . . . ,gk € we define the superposition 

ofuj and gi, . . . ,gk, to be the function ijj[gi, . . . ^gk\ ■ C^^^ — > Q defined by 

uj[gi,...,gkm = E ^(/)- (1) 

flgi,---,9k]=f' 

Example 5.3. Let D be a totally ordered set D, and let C = Clone({max}) where max is the binary 
maximum operation on D. Note that C^^-* contains just three binary operations: e[^\ 63^^ and max. 
Let u) be the 2-ary weighting of C given by 



-1 if/ = ef) 
+1 if/ = e(') 
if / = max 



and let 

.(2) 



(91,52) = %max^ . 



Note that e\ [91, 52] = gi = ^2 ^""^ ^2 bii 92] = 92 = max, so, applying Definition 5.2, we have 

w[9i,52](/) = 



if / = ef ^ 



-1 if/ = e(') • 
+ 1 if / = max 

Note that a;[gi, 52] satisfies the conditions of Definition 5.1 and hence is a 2-ary weighting ofC. 

Example 5.4. Let C be a clone on some totally ordered set D and let max and min be binary maximum and 
minimum operations which are contained in C. Note that C^^^ contains operations such as max[e-^\ ej^''] 
which returns the maximum of the ith and jth argument values. Operations of this form will be denoted 
max(a;j, xj). 

Let u) be the A-ary weighting of C given by 



^ ^ - 1 if / is a projection, that is, / G {e^'*'' , 62"^-* , 63'*'' , ef'' } 
^U) = +1 if / G {max(xi,X2),min(xi,X2),max(x3,X4),min(x3,X4)} 
otherwise 



and let 



(.i,.2,.3,.4) = (ep\ef,er,max(.„X2; 
Then, by Definition 5.2 we have 

{—1 if / is a projection, that is, / G {e^'^^ , 63^^ , eg'^'' } 
+1 if / G {max(xi,X2,X3),min(xi,a;2),min(x3,max(xi,X2))} • 
otherwise 

Note that w [51, (72, 53, 54] satisfies the conditions of Definition 5.1 and hence is a 3-ary weighting of C. 
Example 5.5. Let C and oj be the same as in Example 5.4 but now consider 

(51,92,53:94) = <^e^^\max(x2,X3),min(x2,X3),e^'^^ 
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By Definition 5.2 we have 



' -1 if/G{e 
[51,521 53 > 54] (/)= S +1 if/Gi 



\ %max(x2,X3),min(x2,X3),e^ 
max(2;i, 2;2, 2:3), mm(xi, max(x2, X3)) 
max(min(x2, 2:3), X4), min(3;2, 2^3, 2;4) 



otherwise 



'Note that uj [g'^ ,92,93, 54] does not satisfy the conditions of Definition 5. 1 because, for example, we have 
that i^[5i,52>53'54](/) < when f = max(x2,X3), which is not a projection. Hence ijj[g'i, g'2, d'z, 9'^ 
not a valid weighting ofC. 

It follows immediately from Definition 3.7 that the sum of the weights in any superposition u}[9i, . . . ,9k\ 
is equal to the sum of the weights in oj, which is zero, by Definition 5.1. However, as we have seen in Exam- 
ple 5.5, it is not always the case that an arbitrary superposition satisfies the other condition in Definition 5.1, 
that negative weights are only assigned to projections. Hence we make the following definition: 

Definition 5.6. If the result of a superposition is a valid weighting, then that superposition will be called a 
proper superposition. 

Remark 5.7. The superposition of a projection operation and other projection operations is always a pro- 
jection operation. So, by Definition 5.2, for any clone C and any uj G w[^^\ if 91, . . . , gk G C^^^ are 
projections, then the function uj[gi, . . . , 9^] can take negative values only on projections, and hence is a 
valid weighting. This means that a superposition with any list of projections is always a proper superposi- 
tion. 

We are now ready to define weighted clones. 

Definition 5.8. A weigiited clone, W, is a non-empty set of weightings of some fixed clone C which is 
closed under non-negative scaling, addition of weightings of equal arity, and proper superposition with 
operations from C. The clone C is called the support ofW. 

Example 5.9. For any clone, C, the set Wc containing all possible weightings of C is a weighted clone 
with support C. 

Example 5.10. For any clone, C, the set containing all zero- valued weightings of C is a weighted 
clone with support C. Note that contains exactly one weighting of each possibly arity, which assigns 
the value to all operations in C of that arity. 

We now establish a link between weightings and weighted relations, which will allow us to link 
weighted clones and weighted relational clones. 

Definition 5.11. Let g be a weighted relation of arity r on some set D and let u be a k-ary weighting of 
some clone of operations C on the set D. 

We say that uj is a weighted polymorphism of g if, for any xi , X2, . . . , G D^' such that g{xi) is 
defined for i = 1, . . . ,k, we have that g{f{xi,X2, ■ ■ ■ , x^)) is defined for all f € C^^\ and 



If 00 is a weighted polymorphism of g we say g is improved by uj. 

Note that, by Definition 3.9, if g is improved by the weighting u € w|^\ then every element of C^^^ 
must be a polymorphism of g. 




(2) 



/ec('=) 
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Example 5.12. Consider the class of submodular functions [^ ^]. These are precisely the functions g 
satisfying 

£»(mm(xi,X2)) + ^(max(xi,X2)) - g(xi) - g(x2) < 0. 

In other words, the set of submodular functions are the set of weighted relations with a 2-ary weighted 
polymorphism oJsub, defined by: 

^subij) — +1 if / G {min(xi, X2), max(xi, X2)} ■ 
otherwise 

Submodular function minimisation is known to be tractable [''^]. 

Definition 5.13. For any T C ^£,, we denote by wPol(r) the set of all weightings ofPol(T) which are 
weighted polymorphisms of all weighted relations g (z T. The set ofk-ary weightings in wPol(r) will be 
denoted wPol('')(r). 

To define a mapping in the other direction, we need to consider the union of the sets Wc over all 
clones C on some fixed set D, which will be denoted W/j. If we have a set C W/j which may contain 
weightings of different clones over D, then we can extend each of these weightings with zeros, as necessary, 
so that they are weightings of the same clone C, given by 



C = Clone( (J dom(w)). 



This set of extended weightings obtained from W will be denoted W. For any set C W/j, we define 
wClone(VF) to be the smallest weighted clone containing W. 

Definition 5.14. For any W C W/j, we denote by Imp(H^) the set of all weighted relations in which 
are improved by all weightings ui G W. The set of r-ary weighted relations in \mp{W) will be denoted 
ImpW(VF). 

It follows immediately from the definition of a Galois connection [9] that, for any set D, the mappings 
wPol and Imp form a Galois connection between W/3 and as illustrated in Figure 1 . A characterisation 
of this Galois connection for finite sets D is given by the following two theorems, which are proved in 
Section 6. 

Tlieorem 5.15. For any finite set D, and any finite T Q Imp(wPol(r)) = wRelClone(r). 

Theorem 5.16. For any finite set D, and any finite W C Wo, wPol(Imp(T^)) = wClone(II^). 

As with any Galois connection [9], this means that there is a one-to-one correspondence between 
weighted clones and weighted relational clones. Hence, by Proposition 4.5, Theorem 2.8, and Theorem 4.3, 
the search for tractable valued constraint languages over a finite set corresponds to a search for suitable 
weighted clones of operations. 

6 Proofs of Theorems 5.15 and 5.16 

Our proofs of Theorems 5.15 and 5.16 will both use the following result, which is a variant of the well- 
known Farkas' Lemma used in linear programming [ , ]. 
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Figure 1: Galois connection between and W^i. 
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Lemma 6.1 (Farkas 1894). Let S and T be finite sets of indices, where T is the disjoint union of two 
subsets, r> and T=. For all i € S*, and all j € T, let Uij and bj be rational numbers. Exactly one of the 
following holds: 

• Either there exists a set of non-negative rational numbers {xi | i € S*} and a rational number C such 
that 

for each j £ T>, 
for each j S T= , 

• Or else there exists a set of integers {yj \ j S T} such that "^j^x Vj ~ ^ 

for each j €z T>, 
for each i £ S, 



Such a set is called a certificate of unsolvability. 

We note that there is an effective procedure to decide which of the cases mentioned in Lemma 6. 1 holds for 
any instance, and to calculate the values of the corresponding coefficients Xi or yj [35]. 

We will prove Theorem 5.15 in two parts. First, we show in Proposition 6.2 that the set of all weighted 
relations improved by any given set of weightings is always a weighted relational clone. Then we show 
that for any finite set P the set of weighted relations improved by all weightings in wPol(r) is precisely the 
weighted relational clone wRelClone(r). 

Proposition 6.2. For any finite set D, and any W C W^i, Imp(VF) is a weighted relational clone. 

Proof. Certainly Imp(l^) contains the weighted equality relation, q=, since g= satisfies inequality (2) in 
Definition 5.11 for all Xj such that q= (xj) is defined. Similarly, Imp(TI^) is closed under non-negative scal- 
ing, addition of constants, addition and rearrangement of arguments, since all of these operations preserve 
inequality (2). Hence, to show Imp(VF) is a weighted relational clone we only need to show Imp(VF) is 
closed under minimisation. 

Let g G Imp^''^ (W) and assume that g' is obtained from g by minimising over the last argument. In 
other words, g'{xi,X2, • • • , Xr-i) = mmx^{g{xi,X2, • • • , Xr)). We will now show that g' G Imp(l^). 

Let io £ W be a. k-aiy weighting of a clone C. Since g S Imp(Ty), we know that g and to satisfy 
inequality (2) for all xi, X2, . . . , x^ such that g{^i) is defined. Now consider any x'^^, Xg, . . . , x'^. for which 
each g'i'x.'-) is defined. Extend each x^ to a tuple x" of arity r in such a way that ^>(x") is minimised. Since 
all negative values of uj are associated with projections, we have 

^ u;(/)^>'(/(x;,x'2,...,x',)< ^ ^(/)^>(/(x;',x'2',...,x',')<0. 

□ 

We now prove Theorem 5.15, which states that that for any finite set D, and any finite F C 
Imp(wPol(r)) = wRelClone(r). 



i&S 

^ ^ CLiJ Xi = hj -\- C. 

iPS 



Vj > 0, 

yj Uij < 0, and 

J^yjbj > 0. 
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Proof. We first establish that for any F C $d we have the inclusion wRelClone(r) C Imp(wPol(r)). 
To see this, observe that T C Imp(wPol(r)) and, hence, wRelClone(r) C wRelClone(Imp(wPol(r))) 
which is equal to Imp(wPol(r)) by Proposition 6.2. 

We will prove the reverse inclusion, Imp(wPol(r)) C wRelClone(r), as follows. Given a weighted 
relation q of arity r, we will show that either there exists a weighted operation u) E wPol(r) such that 
Q Imp({tj}) or else q plus some constant can be obtained by minimisation from a non-negative weighted 
sum of weighted relations in F, and hence q € wRelClone(F). 

We now give the details of this argument. Let k be the number of r-tuples for which q is defined and fix 
an arbitrary order, xi, . . . , x^., for these tuples. This list of tuples can be viewed as (the rows of) a matrix 
with k rows and r columns, which we will call S^. 

By Proposition 4.5, q G wRelClone(F) if and only if it can be expressed using weighted relations from 
F^. By Definition 2.5, a weighted relation g' is expressible over F^ if and only if there exists an instance 
V G VCSP(F^) and a hst L of variables of V such that -KLiV) = q' . 

We consider instances V with \D^\ variables, where each variable is associated with a distinct tuple 
from D^. Each constraint of "P is a pair (S, 7) for some 7 G F^ and some list of variables S. Each such S 
can be viewed as a list of A;-tuples over D, and hence as a matrix over D, whose columns are these A;-tuples. 
Since we are using V to express the defined values of g, it is sufficient to consider only matrices S with 
rows ti, . . . , tfc such that 7(tj) is defined for i = 1, . . . , /c. For any 7 G F, a pair {S, 7) with this property 
will be called a A;-match to F. 

Each assignment to the variables of V can be seen as a mapping from A;-tuples over D to D, and hence 
associated with an operation / G O^^. For any list of variables S of V, we will write f{S) to denote the 
assignment to those variables obtained by applying / to the columns of S, viewed as a matrix. With this 
notation, we have that itl{'P) = g' with L = Sgif we can find non-negative rationals xs,y for all A;-matches 
to F, and a constant c G Q, such that the following system of inequahties and equations is satisfied: 



E E ^s,ylifiS)) > g{fiS,)) + c 

7Gr {all fe-matches (5, 7)} 



For each / G , 



E E ^^'-^ 7(e(5')) = ^'(e(S'e)) + c 

7Gr {all fc-matches {S, 7)} 

Moreover, if / ^ Pol'-'^) (F) then the left-hand-side of the corresponding inequality will be undefined, by 
Definition 3.9, so it is sufficient to consider only / G Pol^'^^(F). This gives us a system of inequalities and 
equations with rational coefficients. 

If this system has a solution then g G wRelClone(F). On the other hand, if this system of equations 
and inequalities has no solution, then we appeal to Lemma 6.1, to get a certificate of unsolvability. That is, 
in this case we know that there exists a set of integers {yf \ f G Pol'-'^^ (r)}, such that J2f^Poi('')(r) Vf ~ ^' 
yf < only if / is a projection, and: 

foreach/c-matching (5,7) of F, ^ y/ 7(/(5')) < 0, and (3) 

/gPoi('='(r) 

E yf eifiSo)) > (4) 

/GPoi(*)(r) 



(k) 

For each projection e G O}^ , 
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Now, consider the k-ary weighting u of the clone Pol(r) defined by to{f) = yj for each / G Pol^'^^(r). 
From (3), we can see that u; is a weighted polymorphism of every 7 G F. On the other hand, (4) shows that 
Lo is not a weighted polymorphism of g. □ 



Remark 6.3. The proof of Theorem 5.15 demonstrates the decidability of the following question: for any 
finite F C and any weighted relation g defined on D, does g belong to wRelClone(F) ? 

We will prove Theorem 5.16 in two parts. First, we show in Proposition 6.4 that the set of weighted 
polymorphisms of any given set of weighted relations is always a weighted clone. Then we show that 
for any finite set W the set of weightings that improve all weighted relations in Imp(VF) is precisely the 
weighted clone wClone(VF). 

Proposition 6.4. For any finite set D, and any F C wPo1(F) is a weighted clone. 

Proof. By Definition 5.13, wPol(F) is a set of weightings of Pol(F). Similarly, wPol(F) is closed under 
addition and non-negative scaling, since both of these operations preserve inequality (2). Hence, to show 
wPol(F) is a weighted clone we only need to show wPol(F) is closed under proper superposition by 
members of Pol(F). 

Let io G wPol('')(F) and suppose to' = uj[gi, . . . , 17^] is aproper superposition of cj, where gi,g2, ■ ■ ■ , gu G 
Pol(^)(F). We will now show that u' G wPol(^)(F). Suppose ^) is a weighted relation of arity r satisfying 
oj G wPol({£)}), i.e., g and u; satisfy inequality (2) for all xi, X2, • • • , such that each g{^i) is defined. 
Given any x'^^, Xg, . . . , x^ for which each i?(x') is defined, set Xj = (7j(x'^, Xg, . . . , x^) for i = 1, 2, . . . , /c. 
Then, if we set /' = . . . , 5^.], we have /'(x^, x'2, . . . , x^) = /(xi, xg, . . . , x^), for any / G Pol(^)(F). 
Hence, by Definition 5.2, we have 

u;'{f')g{f'{^[ , x'2, . . . , x^)) = X2, . . . , x,)) < . 

/'gPoiW{r) /ePoi(^)(r) 

□ 

We will make use of the following technical lemma, which shows that any weighted sum of arbitrary 
superpositions of a pair of weightings uji and 002 can be obtained by taking a weighted sum of superpositions 
of uji and uj2 with projection operations, and then taking a superposition of the result. This result implies 
that any weighting which can be expressed as a weighted sum of arbitrary superpositions can also be 
expressed as a superposition of a weighted sum of proper superpositions. 

Lemma 6.5. Let C be a clone, and let ui and uj2 be weightings ofC, of arity k and £ respectively. For any 
gi,...,gke C7(-) and any g[,...,g',e C^^\ 

ciuji[gi,. . . ,gk] + C2UJ2[g'i,. . . ,g'i\ = uj[gi, . . . , gk, g'l, ■ ■ ■ , g'^] , (5) 

where uj = ci uji[e\ ,...,e^ 'J + C2 a;2[e^_^_/, . . . , e^^^ 'J 

Proof. For any / G C^"^\ the result of applying the right-hand side expression in equation (5) to / is: 

/ \ 



E 



f'lgi,-,9k,g'i,-,ge]=f \^h'[i 



h'[e 



{k+i) 
k + 1 ' 



]=/' / 
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Replacing each /' by the equivalent superposition of h' with projections, we obtain: 



h' 



cia;i(/i') + 

[9\,---,9k\=f 



22 C2UJ2{h') 



which is the result of applying the left-hand-side of Equation 5 to /. 



□ 



We now prove Theorem 5.16, which states that for any finite set D, and any finite W C W^), 
wPol(Imp(VF)) = wClone(W^). 

Proof. We first establish that for any W C W/j we have the inclusion wClone(M^) C wPol(Imp(Ty)). 
To see this, observe that every operation in C = Clone(U^g^^ dom.{ijj)) is a polymorphism of Imp(V7), 
by Definition 3.9, so 1^ C wPol(Imp(VF)). Hence, wClone(VF) C wClone(wPol(Imp(l^))) which is 
equal to wPol(Imp(T4^)) by Proposition 6.4. 

We will prove the reverse inclusion, wPol(Imp(l^)) C wClone(VF), as follows. Given any weighting 
ixiQ € W/), we will show that either there exists a weighted relation q G Imp(VF) such that wq wPol({^?}) 
or else is equal to a non-negative weighted sum of superpositions of weightings in W, and hence ujq G 
wClone(Ty). 

We now give the details of this argument. Let k be the arity of ljq, and let M = \D\^. We first observe 
that it is sufficient to consider weighted relations of arity M in Imp(l^). To see this, suppose there exists 
a weighted relation q G Imp(VF) with arity N > M such that loq wPol({£)}) and let xi, . . . , G 
be any set of tuples for which inequality (2) fails to hold for ujq and g. Let X be the k x N matrix whose 
rows are the tuples xi, . . . , x^. Since > M it follows that some of the columns in this matrix must be 
equal. Moreover, if the i-th and j-th column of X are equal, then so will be the i-th and j-th entry of the 
tuple /(xi, . . . , Xfc) obtained by applying any / G O^^ to these k tuples. 

Now let g' be the weighted relation of arity < M that depends only on the first of each of these repeated 
columns, and takes the same values as g takes on arguments with the appropriate entries repeated. Let X' 
be the reduced form of X (with repeated columns deleted). By this approach, we can construct g' so that 
g' G Imp(VK), but X' gives a certificate to show that ujq wPol({g}), i.e., the rows of X' form a list of 
tuples for which (2) fails to hold for loq and g'. 

Moreover, if we have a weighted relation g G Imp(VF) with arity N < M such that loq wPol({£)}), 
then g can be extended to a weighted relation g' of arity M that does not depend on the M — N added 
inputs, and, hence, is also contained in Imp(Ty) and is such that loq wPol({^>'}). 

By the argument given above, there exists a weighted relation g G Imp(l^) such that loq wPol({£)}) 
if and only if there exists a weighted relation gM of arity M in Imp(l^) such that ujq wPo1({^>m})- 
Furthermore, by reordering the arguments of gM if necessary, we can assume that gM and loq violate (2) 
on the particular list of tuples xi, . . . , x^ given by taking the rows of a matrix, Xj\f , whose columns are 
precisely the tuples in D^, ordered lexicographically. 

By Definition 5.11, such a weighted relation gM exists if and only if the following system of in- 
equalities can be satisfied, for all uj £ W and all ti, . . . ,tai.(a;) ^ such that gMi'^i) is defined for 
i = 1, . . . , ar(a;). 




tarn)) < 



(6) 



and, for the tuples xi 



. . . , Xfc forming the rows of Xm, Qui'^i) is defined for z = 1, . . . , A; and 





(7) 



/edom(ajo) 
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There is a one-to-one correspondence between operations g : — > D and tuples tg G , where the 
tuple tg contains the list of values returned by the operation g applied to the columns of Xm- 

Set C = Clone(|J^g^y dom(a;)). We observe that, to satisfy inequality (6), for any uj € W , if QM{^i) 
is defined for i = 1, . . . , ar(a;), then Quigi^i, ■ ■ ■ , tar{w))) must be defined for all g G dom(a;). To 
achieve this, it is sufficient to ensure that, for all g G C^'^K Qui^g) is defined. All other values of qm can 
be left undefined, as this just reduces the number of inequalities in the system. 

Using superposition (Definition 5.2), we can rewrite inequalities (6) to obtain the following equivalent 
system: for all uj G W, and all 51, ... , gs.r[u)) G C^'^\ 

)](/) . . . ,Xfc)) < 0. (8) 

ar(a;)]) 

Now, by applying Lemma 6. 1 to the resulting system of inequalities, we conclude that either a solution 
Q]\j exists, in which case loq ^ wPol(Imp(Ty)), or else there exists a set of non-negative rational numbers 

{c-[3i....,9a.M] \^<^W,gi,.. . e C^'^)} 

such that for every / G C^''\ 

J2_ Yl C^[3i,...,<;a.M]^bl.---'5arM](/) >^0(/)- (9) 

(5l,.--,Sar(c^)) 

By Definition 5.1, adding the left-hand side of these inequalities over all / gives 0, and so does adding 
the right hand sides, so each inequality must actually be an equality. In other words, cjq is equal to a non- 
negative weighted sum of superpositions of weightings in W. Hence, by Lemma 6.5 and Remark 5.7, coq 
is equal to a proper superposition of some element Wq G wClone(M/^), so ujq G wClone(M^). □ 

Remark 6.6. The proof of Theorem 5.16 demonstrates the decidability of the following question: for any 
finite W C W^i and any weighting uj defined on D, does uj belong to wClone(VF).'' 



7 Necessary Conditions For Tractability 

In this section, we will start to investigate the structure of weighted clones, and hence establish some 
necessary conditions for any valued constraint language to be tractable. 

Note that, by Definition 3.8, the smallest possible clone of operations over a fixed set D is the set of all 
projection operations on D, which is denoted Jd. 

Proposition 7.1. For any finite set D, there are precisely two weighted clones with support Jjy. These are 
Wj^ and W^j^. 

Proof. Let be a weighted clone with support 3d- 

If the weights assigned by every weighting a; G are all zero, then W is the zero-valued weighted 
clone Wj^ described in Example 5.10. 

Otherwise, there is some uj £ W (of arity k) such that uj assigns positive weight to some /c-ary projec- 
tions and negative weights to some of the others (the sum of the weights is zero, by Definition 5.1). If we 
form the superposition of uj with the sequence of projections gi, g2, ■ ■ ■ , gk^ where gi = e^a^ if uj{e"^^) is 
positive, and gi = e^^^ otherwise, then we obtain a new weighting uj[gi, g2, ■ ■ ■ , gk] 3 d which assigns 

(k) Ik) 

some positive weight wio eh, and —wioel . 
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By adding appropriate multiples of such functions for each successive pair of indices a and b, we can 
obtain any desired weighting of J/). Hence, in this case W contains all possible weightings of J/), so 



Any weighting w which is defined only for projection operations will be called a trivial weighting, 
Proposition 7.2. For any set of trivial weightings W, Imp(PF) is NP-hard. 

Proof. By Theorems 5.15 and 5.16, we have that Imp(wClone(VF)) = wRelClone(Imp(l^)), so by 
by Proposition 4.5, Theorem 2.8, and Theorem 4.3 we have that Imp(VF) is NP-hard if and only if 
Imp(wClone(VF)) is NP-hard. 

If W contains only trivial weightings, then wClone(VF) has support J £,, so it is equal to Wj^ or Wj^ 
by Proposition 7.1. 

Every weighting in Wj^ is a weighted polymorphism of any possible weighted relation, by Defini- 
tion 5.11. Hence Imp(W5^) = *d. 

The weighted relations that are improved by all weightings are precisely those which take at most one 
value. Hence Imp(Wj^) = (R/))~. 

In both cases the resulting valued constraint language is NP-hard. □ 

Now consider weightings whose values are all 0. 
Proposition 7.3. For any set of zero-valued weightings W, Imp(VF) is NP-hard. 

Proof. By Definition 5.11, a zero-valued weighting will be a weighted polymorphism of any weighted 
relation which is a total function (i.e., any weighted relation where all assignments are feasible). Some 
valued constraint languages containing only total functions are NP-hard [ ]. For example, consider the 
valued constraint language consisting of the following total function: 



We observed in Example 2.3 that on the domain {0, 1} the problem VCSP({£)^}) corresponds to the Max- 
CUT problem which is known to be NP-hard. Over domains of size k > 2 this problem corresponds to the 



Using the Galois connection developed in the previous sections, these two results tell us that any val- 
ued constraint language that is not NP-hard must have a weighted polymorphism which is non-trivial and 
assigns at least some non-zero weights. A weighting which assigns positive weight to at least one operation 
that is not a projection will be called a positive weighting. 

Corollary 7.4. For any finite set D, and any T C ^d, either T is NP-hard, or else wPol(r) is a weighted 
clone containing some positive weightings. 

Proof. By Proposition 6.4, in all cases wPol(r) is a weighted clone. 

By Theorem 5.15, for any finite T' C T, Imp(wPol(r')) = wRelClone(r'). By Proposition 4.5, 
Theorem 2.8, and Theorem 4.3, if T' is NP-hard, then wRelClone(r') is also NP-hard, so Imp(wPol(r')) 
must be NP-hard. 

Conversely, if V' is not NP-hard, then the same argument shows that Imp(wPol(r')) is not NP-hard, 
so by Propositions 7.2 and 7.3, wPol(r') must contain some weightings that are non-trivial and some 
weightings that are not zero-valued. 




problem Max-/c-Cut, which is also known to be NP-hard. 



□ 
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Choose a weighting a; G wPol(r) that is not zero-valued, and a weighting uj' that is non-trivial (but 
may be zero-valued). If uj assigns positive weight to any non-projection, then it is a positive weighting and 
we are done. 

Otherwise, we have that uj assigns positive weight to some projections and negative weight to some 
other projections. Let / be an operation on which w' is defined that is not a projection, and let k be the arity 
of /. If we form the superposition of uj with the sequence of functions gi,g2, ■ ■ ■ , 5ar(w)> where gi = f if 
i^^g('ir(<^))-j positive, and gi = e^^ ^ otherwise, then we obtain a new weighting Lo[gi, g2, ■ ■ ■ , 5ar(a;)] which 
assigns positive weight to / (see Example 5.3). □ 

Assuming that P ^ NP, this result tells us that tractable valued constraint languages are associated 
with certain kinds of weighted clones. 

To obtain further information about the weighted clones associated with tractable valued constraint 
languages, we now consider some special kinds of operations. For any > 2, a fe-ary operation / is called 
sharp if / is not a projection, but the operation obtained by equating any two inputs in / is a projection [20]. 
In other words, / is sharp if for alH, j G {1, . . . , fc — 1} with i ^ j, there exists an index m € {1, . . . , A; — 1} 
such that / satisfies the identity: f{xi,X2, ■ ■ ■ , Xj^i,Xi, Xj,Xj^i, . . . , x^-i) = Xm- 

Theorem 7.5. Any weighted clone W containing positive weightings must contain a weighting that assigns 
positive weight to either: 

1. A set of unary operations that are not projections; or 

2. A set of sharp operations. 

Proof. Let a; be a positive weighting in W with the smallest possible arity, k. \f k = 1, then we are 

done. Otherwise, we consider the weightings uj[e]' ^\^^2 • • • > ^^j^-i\^^t ^^f^ ^\ ■ ■ ■ ^ ^i-i^''] ^'^^ 
alH,j e{l,...,k} withi 7^ j. 

Each of these weightings has arity A; — 1, so, by the choice of uj, must not assign positive weight to any 
operation except (possibly) projections. Hence all non-projection operations assigned positive weight by lo 
are sharp. □ 

We can obtain further details about these weighted clones by considering the possible types of sharp 
operations. 

First, we observe that all sharp operations must satisfy the identity /(x, x, . . . ,x) = x; such operations 
are called idempotent. 

Ternary sharp operations may be classified according to their values on tuples of the form {x, x, y), {x, y, x) 
and {y, x, x), which must be equal to either x or y. There are precisely 8 possibilities, as listed in Table 1. 



Input 


1 


2 


3 


4 


5 


6 


7 


8 


(x,x,y) 


X 


X 


X 


X 


y 


y 


y 


y 


(x,y,x) 


X 


X 


y 


y 


X 


X 


y 


y 


(y,x,x) 


X 


y 


X 


y 


X 


y 


X 


y 



Table 1 : Sharp ternary operations 

The first column in Table 1 corresponds to operations that satisfy the identities f{x, x, y) = /(x, y, x) = 
f{y,x,x) = X for all x,y ^ D; such operations are called majority operations. The last column in the 
table corresponds to operations that satisfy the identities f{x,x,y) = f{x,y,x) = f{y,x,x) = y for 
all X, y E D; such operations are called minority operations. Columns 4, 6 and 7 in Table 1 correspond 
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to operations that satisfy the identities f{y,y,x) = f{x,y,x) = f{y,x,x) = y for all x,y £ D (up to 
permutations of inputs); such operations are called Pixley operations [ ]. 

For any A; > 3, a k-aiy operation / is called a semiprojection if it is not a projection, but there is 
an index i G {1, . . . , A;} such that /(xi, . . . , x^) = ef^'^ for all xi, ■ ■ ■ ,Xk € D such that xi, . . . , 2;^ are 
not pairwise distinct. In other words, a semiprojection is a particular form of sharp operation where the 
operation obtained by equating any two inputs is always the same projection. Columns 2,3 and 5 in Table 1 
correspond to semiprojections. 

The following lemma shows that the only sharp operations of arity A; > 4 are semiprojections. 

Lemma 7.6 (Swierczkowski's Lemma [ ]). Given an operation of arity > 4, if every operation arising 
from the identification of two variables is a projection, then these projections coincide. 

Hence we may refine Theorem 7.5 to obtain the following corollary. 

Corollary 7.7. Any weighted clone W containing positive weightings must contain a weighting that assigns 
positive weight to either: 

1. A set of unary operations that are not projections; or 

2. A set of binary idempotent operations that are not projections; or 

3. A set of ternary operations that are majority operations, minority operations, Pixley operations or 
semiprojections; or 

4. A set of k-ary semiprojections (for some k > 3). 

Corollary 7.7 can be used to guide the search for tractable valued consti^aint languages, as we illustrate 
in the next section. 

8 Classification of Boolean Valued Constraint Languages 

In this section, we consider the special case of valued constraint languages over a 2- valued domain, such as 
the Boolean domain D = {0,1}. 

There are only four unary operations on the Boolean domain, and one of these is the projection operation 
e^^\ which is the identity operation. The remaining three unary operations are the operations given by 
f{x) = 0, /(x) = 1, and f{x) = 1 — x. These will be referred to as constant 0, constant 1, and inversion. 

There are only two binary idempotent operations on the Boolean domain that are not projections: the 
operations min and max. The only sharp ternary operations are the unique majority operation (which we 
will call Mjrty), the unique minority operation (which we will call Mnrty), and three Pixley operations. 
There are no semiprojections. 

Hence we can refine Corollary 7.7 even further in the special case of the Boolean domain, to limit the 
possibilities for weighted clones associated with tractable valued constraint languages to just nine cases. 

Theorem 8.1. Any weighted clone W on the Boolean domain that contains positive weightings must con- 
tain a weighting lo that assigns positive weight to either: 

1. Exactly one of the unary operations constant 0, constant 1, or inversion; 

2. Exactly one of the binary operations min and max, or both of them equally; 

3. Exactly one of the ternary operations Mjrty and Mnrty, or both of them with c<j(Mjrty) = 2a;(Mnrty). 
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Proof. Let C be the support of W , and let w be a positive weighting in W with minimal possible arity. 
Since there are no semiprojections on the Boolean domain, Corollary 7.7 tells us that uj is either unaiy, 
binary or ternary. 

Consider first the case when oj is unary. Since there are just three unary operations on the Boolean 
domain that are not projections, scale oj so it assigns weight -1 to the projection ej^^'', weight a to the 
constant operation /o, weight b to the constant 1 operation /i, and weight c to the inversion operation /-,. 
If c = 1, then UJ assigns positive weight only to /-, and we are done. Otherwise, if G C, and hence c is 
defined, we consider the weighting uj' = -^u) + -^oj[f^]. It is straightforward to check that w' assigns 

weight c — 1 to e[^\ weight a to /q, weight h to /i and weight to /-,. By Lemma 6.5, uj' belongs to W. 

If a = 1, then uj' assigns positive weight only to /o and we are done. Otherwise, if /q G C, and hence 
a is defined, we consider the weighting uj" = uj' + ]:z^w'[/o]. It is straightforward to check that uj" assigns 
positive weight only to /i. By Lemma 6.5, uj" belongs to W. 

Next consider the case when uj is binary. By Corollary 7.7 and our observations above about the possible 
binary idempotent operations on the Boolean domain, we know that uj assigns positive weight only to one 
or both of the operations min and max. If either of these weights is undefined (because the corresponding 
function does not belong to C), or zero, then we are done, so assume that uj assigns positive weight to both 
min and max. By taking the weighting uj + uj[e2 , ], with a suitable scaling, we can obtain a weighting 
uja ^ W that assigns weight -1 to both binary projections, weight a to min and weight 2 — o to max, for 
some < a < 2. 

If a < 1, then the weighting uJa + Y^a;a[niin, max] assigns positive weight only to max. If a > 1, 
then the weighting uja + |5f '^a[iiiin, max] assigns positive weight only to min. If a = 1, then uj assigns 
equal weight to min and max. 

Finally, we consider the case when uj is ternary. By Corollary 7.7 and our observations above about 
the possible ternary sharp operations on the Boolean domain, we know that uj assigns positive weight to 
some subset of Mjrty, Mnrty and the three Boolean Pixley operations /i, /2 and /s (corresponding to the 
fourth, sixth and seventh columns of Table 1). We note that /i [63'^^ , eg^'' , e^^^] = /s, /2 [62^^ , ef^ , ef^] = /i 
and /3[e2^'', 63^^ e^^''] = /2. Hence, if uj assigns positive weight to any Pixley operation, then we have that 

(3) f3) f3^ f3) f3) (3") 

W also contains the weighting uj' = uj + uj[&2 ,63 , e]^ ] + w [eg , ,63 ] which assigns equal negative 
weight to each projection, and equal positive weight to each Pixley operation. By a suitable scaling we 
shall assume that uj' assigns weight - 1 to each projection. 

Suppose first that uj' assigns positive weight to at least one of Mjrty and Mnrty, and assigns weight 
< w < 1 to the three Pixley operations. We note that /2, /3] = for each i = 1, 2, 3. Moreover, 
Mjrty[/i, /2, h] = Mnrty and Mnrty[/i, /2, h] = Mjrty. Thus, the weighting uj" = uj' + wuj'[fi,f2, fs] 
is non-zero, assigns weight to each Pixley operation and equal negative weight to all projections. By 
Lemma 6.5, lj" G W. 

Assume that u" assigns positive weight to both Mnrty and Mjrty. By taking a suitable scaling, we can 
obtain a weighting uja ^ W that assigns weight -1 to all three projections, weight a to Mnrty and weight 
3 — a to Mjrty, for some < a < 3. 

If a < 1, then the weighting + j^Wa [Mjrty, Mjrty, Mnrty] assigns positive weight only to Mjrty. 
If a > 1, then the weighting uja + ^5f'^a [Mjrty, Mjrty, Mnrty] assigns positive weight only to Mnrty. If 
a = 1, then uJa assigns positive weight to both Mjrty and Mnrty, in the ratio 2:1. 

The only remaining case is when uj' assigns positive weight I just to the three Pixley operations. In 
this case we note that /i[ef \ e^^\ /i] = Mjrty, /2[ef \ e^^\ /i] = ei and /3[ef \ e^^\ /i] = 62- Thus the 
function fii = ^^[ei, e2, /i] assigns weight —1 to /i, +1 to Mjrty, and otherwise. For i = 2,3, we can 
obtain in a similar way a function ^i, which assigns weight — 1 to /j and +1 to Mjrty. Then the weighting 
cj + /Ui + ^2 + At3 will assign positive weight only to Mjrty. Again, by Lemma 6.5, u" G W. □ 
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Each of the nine types of weightings mentioned in Theorem 8.1 can be supported by a different clone, 
so these nine types of weightings can each generate different weighted clones. 

Using the Galois connection developed above, this result tells us that any tractable valued constraint 
language over the Boolean domain must have as a weighted polymorphism one of nine specific kinds of 
weightings. Eight of these can be been shown to be sufficient to ensure tractability using the results of [16]. 
The only remaining case is the unary weighting that assigns positive weight to the inversion operation only. 
Our next result shows that having a weighted polymorphism of this kind is not a sufficient condition for 
tractability on its own, but if a language has any additional positive weightings as weighted polymorphisms 
which are not implied by this one, then it will be tractable. 

Corollary 8.2. Any weighted clone W on the Boolean domain that contains positive weightings, satisfies 
exactly one of the following: 

1. W = wClone({a;^} U W^),/or some unary weighting uj^ that assigns positive weight to the inver- 
sion operation only, where C is the support ofW; in this case Imp(VF) is NP-hard. 

2. W contains one of the eight other kinds of weighting listed in Theorem 8.1; in each of these eight 
cases lmp(W) is tractable. 

Proof. By Theorem 8.1, must contain either a weighting uj^ that assigns positive weight to the inversion 
operation only, or at least one of the eight other kinds of weighting listed in Theorem 8.1 (or both). 

\f W = wClonedtj-,} U W^), for some unary weighting uo^ that assigns positive weight to the 
inversion operation only and C is the support of W, then we are in case (1). In this case, the weighted 
relation defined in Example 2.3 is an element of Imp(PF), so Imp(M^) is NP-hard (see the proof of 
Proposition 7.3). 

If W contains a suitable weighting uo^, but W ^ wClone({a;^} U W^), then W must also contain a 
non-zero weighting uj of minimal possible arity such that uj wClone({tj^} U W^). 

If oj is unary, then we can use the same argument as in the proof of Theorem 8.1 to show that W must 
contain a unary operation that assigns positive weight to the constant 1 operation only or the constant 
operation only. 

If uj is not unary, then <x'[e[^\ . . . , must lie in wClone(a;-,), so uj assigns positive weights only to 
operations / such that /(x, . . . , x) = x or /(x, . . . , x) = 1 — x. If a;(/) = a > for some / such that 
/(x, . . . ,x) = 1 — X, then we consider the weighting oj' = uj + aijj^[f], and note that u}'{f) = 0. By 
repeating this process we obtain a weighting oj" which assigns positive weight only to operations / such 
that /(x, . . . , x) = X. Since u) has minimal arity, these must be sharp operations, so we can proceed as in 
the proof of Theorem 8.1 to show that case (2) holds. 

Each of the eight types of weightings in case (2) is sufficient to ensure the tractability of Imp(M^), by 
the results of [16]. □ 

The corresponding classification for valued constraint languages over the Boolean domain was obtained 
in [16] using a more intricate argument involving the explicit construction of gadgets to express particular 
weighted relations. Here we have considered only the properties of weighted clones. 

Example 8.3. The weighted relation q= defined in Example 2.2 has as a weighted polymorphism the 
weighting uJsub defined in Example 5.12 which assigns equal positive weight to max and min. 

Hence the valued constraint language T = {q=} is tractable, and remains tractable if we add to T any 
other weighted relations that have this weighting as a weighted polymorphism. For example, we may add 
unary weighted relations with a single allowed value, which allow us to fix individual variables to a desired 
value, and still retain tractability. 
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Example 8.4. The weighted relation defined in Example 2.3 has a unary weighted polymorphism that 
assigns positive weight only to the inversion fimction. It has none of the other eight types of weightings 
listed in Theorem 8.1. 

It follows that Express({^=}). 

9 Conclusions 

We have presented an algebraic theory of valued constraint languages that generalizes and extends the alge- 
braic theory developed over the past few years to study the complexity of the classical constraint satisfaction 
problem. We have shown that the complexity of any valued constraint language over a finite domain with 
rational-valued costs is determined by certain algebraic properties which we have called weighted poly- 
morphisms. 

When the weights are all zero, the optimisation problem we are considering collapses to the classi- 
cal constraint satisfaction problem, CSP. In previous work [28, 27] it has been shown that every tractable 
constraint language for the CSP can be characterised by an associated clone of operations. That work ini- 
tiated the use of algebraic properties in the search for tractable constraint languages, an area that has seen 
considerable activity in recent years; see, for instance, [1 1, 15, 14, 13, 29, 21, 4, 2, 3, The results in 
this paper show that a similar result holds for the valued constraint satisfaction problem: every tractable 
valued constraint language is characterised by an associated weighted clone. We therefore hope that our 
results here will provide a similar impetus for the investigation of a much broader class of discrete opti- 
misation problems. For example, a recent result on the power of linear programming for valued constraint 
languages [38] provides a characterisation of languages solvable by a standard LP relaxation in terms of 
weighted polymorphisms. 

Many questions about the complexity of discrete optimisation problems over finite domains can now be 
translated into questions about the structure of weighted clones. This provides a new approach to tackling 
such questions by investigating the algebraic properties of weighted clones. 

References 

[1] Libor Barto. The dichotomy for conservative constraint satisfaction problems revisited. In Proceed- 
ings of the 26th IEEE Symposium on Logic in Computer Science (LICS'll), pages 301-310. IEEE 
Computer Society, 2011. 2 

[2] Libor Barto and Marcin Kozik. Constraint Satisfaction Problems of Bounded Width. In Proceedings 
of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS'09), pages 461- 
471. IEEE Computer Society, 2009. 2, 7, 23 

[3] Libor Barto, Marcin Kozik, Miklos Maroti, and Todd Niven. CSP dichotomy for special triads. 
Proceedings of the American Mathematical Society, 137(9):2921-2934, 2009. 2,7,23 

[4] Libor Barto, Marcin Kozik, and Todd Niven. The CSP dichotomy holds for digraphs with no sources 
and no sinks (a positive answer to a conjecture of Bang-Jensen and Hell). SIAM Journal on Comput- 
ing, 38(5): 1782-1802, 2009. 2, 7, 23 

[5] Joel Berman, Pawel Idziak, Petar Markovic, Ralph McKenzie, Matthew Valeriote, and Ross Willard. 
Varieties with few subalgebras of powers. Transactions of the American Mathematical Society, 
362(3): 1445-1473, 2010. 2, 7, 23 



23 



[6] Stefano Bistarelli, Ugo Montanari, and Francesca Rossi. Semiring-based Constraint Satisfaction and 
Optimisation. Journal of the ACM, 44{2y.20\-236, 1997. 1 

[7] Stefano Bistarelli, Ugo Montanari, Francesca Rossi, Thomas Schiex, Gerard Verfaillie, and Helene 
Fargier. Semiring-based CSPs and Valued CSPs: Frameworks, Properties, and Comparison. Con- 
straints, 4{3yA99-240, 1999. 1 

[8] V.G. Bodnarcuk, L.A. Kaluznin, V.N. Kotov, and B.A. Romov. Galois theory for Post algebras. I. 
Cybernetics and Systems Analysis, 5(3):243-252, 1969. 7 

[9] Ferdinand Borner. Basics of Galois connections. In Complexity of Constraints, volume 5250 of 
Lecture Notes in Computer Science, pages 38-67. Springer, 2008. 7, 11 

[10] Endre Boros and Peter L. Hammer. Pseudo-Boolean optimization. Discrete Applied Mathematics, 
123(l-3):155-225, 2002. 1 

[11] A.A. Bulatov, A.A. Krokhin, and PG. Jeavons. The complexity of maximal constraint languages. In 
Proceedings 33rd ACM Symposium on Theory of Computing (STOC'Ol), pages 667-674, 2001. 2, 7, 
23 

[12] Andrei Bulatov. A Graph of a Relational Structure and Constraint Satisfaction Problems. In Proceed- 
ings 19th IEEE Symposium on Logic in Computer Science (LICS'04), pages 448^57. IEEE Computer 
Society, 2004. 2, 7 

[13] Andrei Bulatov. A dichotomy theorem for constraint satisfaction problems on a 3-element set. Journal 
of the ACM, 53(I):66-I20, 2006. 2, 7, 23 

[14] Andrei Bulatov, Andrei Krokhin, and Peter Jeavons. Classifying the Complexity of Constraints using 
Finite Algebras. SI AM Journal on Computing, 34(3):720-742, 2005. 2, 6, 7, 23 

[15] Andrei A. Bulatov. Complexity of conservative constraint satisfaction problems. ACM Transactions 
on Computational Logic, 12(4):24, 2011. 2, 7, 23 

[16] David A. Cohen, Martin C. Cooper, Peter G. Jeavons, and Andrei A. Krokhin. The Complexity of 
Soft Constraint Satisfaction. Artificial Intelligence, 170(1 1):983-1016, 2006. 1, 2, 3, 5, 18, 22 

[17] Yves Crama and Peter L. Hammer. Boolean Functions - Theory, Algorithms, and Applications . Cam- 
bridge University Press, 2011. 1 

[18] N. Creignou. A dichotomy theorem for maximum generalized satisfiability problems. Journal of 
Computer and System Sciences, 5\{3):5\l— 522, 1995. 2 

[19] Nadia Creignou, Sanjeev Khanna, and Madhu Sudan. Complexity Classification of Boolean Con- 
straint Satisfaction Problems, volume 7 of SIAM Monographs on Discrete Mathematics and Applica- 
tions. SIAM, 2001. 2 

[20] Bela Csakany. Minimal clones - a minicourse. Algebra Universalis, 54(l):73-89, 2005. 19 

[21] Vladimir Deineko, Peter Jonsson, Mikael Klasson, and Andrei Krokhin. The approximability of Max 
CSP with fixed-value constraints. Journal of the ACM, 55(4), 2008. 2, 23 

[22] Tomas Feder and Moshe Y. Vardi. The Computational Structure of Monotone Monadic SNP and 
Constraint Satisfaction: A Study through Datalog and Group Theory. SIAM Journal on Computing, 
28(1):57-104, 1998. 2 



24 



[23] Michael R. Garey and David S. Johnson. Computers and Intractability: A Guide to the Theory of 
NP-Completeness . W.H. Freeman, 1979. 4 

[24] D. Geiger. Closed systems of functions and predicates. Pacific Journal of Mathematics , 27(1):95-100, 
1968. 7 

[25] Gregory Gutin, Pavol Hell, Arash Rafiey, and Anders Yeo. A dichotomy for minimum cost graph 
homomorphisms. European Journal of Combinatorics, 29 (4): 900-911, 2008. 2,4 

[26] Marc Gyssens, Peter G. Jeavons, and David A. Cohen. Decomposing Constraint Satisfaction Problems 
Using Database Techniques. Artificial Intelligence, 66(l):57-89, 1994. 6 

[27] Peter G. Jeavons. On the Algebraic Structure of Combinatorial Problems. Theoretical Computer 
Science, 200(1-2): 185-204, 1998. 2, 6, 7, 23 

[28] Peter G. Jeavons, David A. Cohen, and Marc Gyssens. Closure Properties of Constraints. Journal of 
f/ie ACM, 44(4):527-548, 1997. 2,23 

[29] Peter Jonsson, Mikael Klasson, and Andrei Krokhin. The Approximability of Three-valued MAX 
CSP. SIAM Journal on Computing, 35(6):1329-1349, 2006. 2, 23 

[30] Vladimir Kolmogorov and Stanislav Zivny. The complexity of conservative valued CSPs. In Proceed- 
ings of the 23 rd Annual ACM -SIAM Symposium on Discrete Algorithms (SODA' 12), pages 750-759. 
SIAM, 2012. Full version available on arXiv: 11 10.2809. 2 

[31] Steffen L. Lauritzen. Graphical Models. Oxford University Press, 1996. 1 

[32] George L. Nemhauser and Laurence A. Wolsey. Integer and Combinatorial Optimization. John Wiley 
&Sons, 1988. 3, 11 

[33] A. F. Pixley. Distributivity and permutability of congruence relations in equational classes of algebras. 
Proceedings of the American Mathematical Society, 14(1):105-109, 1963. 20 

[34] Thomas J. Schaefer. The Complexity of Satisfiability Problems. In Proceedings of the 10th Annual 
ACM Symposium on Theory of Computing (STOC'78), pages 216-226. ACM, 1978. 2 

[35] Alexander Schrijver. Theory of linear and integer programming. John Wiley & Sons, Inc., 1986. 11, 
13 

[36] S. Swierczkowski. Algebras which are independently generated by every n elements. Fundamenta 
Mathematicae, 49:93-104, 1960. 20 

[37] Rustem Takhanov. A Dichotomy Theorem for the General Minimum Cost Homomorphism Problem. 
In Proceedings of the 27th International Symposium on Theoretical Aspects of Computer Science 
(STAGS' 10), pages 657-668, 2010. 2, 4 

[38] Johan Thapper and Stanislav Zivny. The power of linear programming for valued CSPs. In Proceed- 
ings of the 53rd Annual IEEE Symposium on Foundations of Computer Science (FOCS' 12). IEEE, 
2012. Full version available on arXiv: 1204. 1079. 3, 23 

[39] Martin J. Wainwright and Michael I. Jordan. Graphical models, exponential families, and variational 
inference. Foundations and Trends in Machine Learning, l(l-2):l-305, 2008. 1 



25 



[40] Tomas Werner. A Linear Programming Approach to Max-Sum Problem: A Review. IEEE Transac- 
tions on Pattern Analysis and Machine Intelligence, 29(7): 1 165-1 179, 2007. 1 

[41] Stanislav Zivny, David A. Cohen, and Peter G. Jeavons. The Expressive Power of Binary Submodular 
Functions. Discrete Applied Mathematics , 157(15):3347-3358, 2009. 2 



26 



